Structured Textbook Review and Individualized Learning Plans Successfully Remediate Underperforming Residents and Improve General Surgery Program Performance on the ABSITE.

Background: We endeavored to create an evidence-based curriculum to improve general surgery resident fund of knowledge. Global and resident-specific interventions were employed to this end. These interventions were monitored via multiple choice question results on a weekly basis and American Board of Surgery In-Training Examination (ABSITE) performance. Methods: This study was performed in a prospective manner over a 2-year period. A structured textbook review with testing was implemented for all residents. A focused textbook question-writing assignment and a Surgical Council on Resident Education (SCORE)-based individualized learning plan (ILP) were implemented for residents scoring below the 35th percentile on the ABSITE. Results: Curriculum implementation resulted in a statistically significant reduction in the number of residents scoring below the 35th percentile, from 50% to 30.8% (P = .023). One hundred percent of residents initially scoring below the 35th percentile were successfully remediated over the study period. Average overall program ABSITE percentile scores increased from 38.5% to 51.4% over a 2-year period. Conclusion: Structured textbook review and testing combined with a question-writing assignment and a SCORE-focused ILP successfully remediated residents scoring below the 35th percentile and improved general surgery residency ABSITE performance.

Online vs in-person delivery of preclinical coursework: A retrospective cohort study evaluating National Board of Chiropractic Examiners performance.

OBJECTIVE: To evaluate the association between basic science curriculum delivery method with other academic and demographic factors on National Board of Chiropractic Examiners (NBCE) part I pass rates. METHODS: This was a retrospective cohort study of students from 3 campuses of 1 chiropractic institution who matriculated in 2018 or 2020. COVID-19 regulations required online delivery of a basic science curriculum for students in the 2020 cohorts, whereas students in the 2018 cohorts experienced a traditional classroom delivery. A general linear model estimated odds ratios for passing NBCE part I, comparing individual online cohorts with the combined classroom cohort while adjusting for academic and demographic variables. RESULTS: A total of 968 students were included, 55% from the classroom cohort. The spring 2020 cohort had the fewest students with bachelors' degrees (59%) and more students with high in-program grade point averages (GPA; 61%) along with the lowest estimated odds ratio [0.80 (95% CI: 0.73-0.87)] for passing vs the classroom cohort. The fall 2020 cohort had significantly higher odds [1.06 (95% CI: 1.00-1.03)] of passing vs the classroom cohort. Additional predictors included main campus matriculation, white ethnicity, bachelors' degree, no alternative admission status, and in-program GPA. Students with high in-program GPA (vs low) had a 36% increased odds of passing. CONCLUSION: Compared to the classroom cohort, the spring 2020 cohort had the lowest odds while the fall 2020 cohort had the highest odds of passing part I. In-program GPA had the highest association with passing. These results provide information on how curriculum delivery impacts board exam performance.

The Effect of Direct Observation of Procedural Skills on Learning and Satisfaction of Students in Community Dentistry Course.

Introduction: Direct Observation of Practical Skills (DOPS) tests is a valuable method for clinical assessment. This study aimed to implement the DOPS test to assess the procedural skills of community dentistry courses and its effects on mastery learning and satisfaction of professors and students at Tabriz faculty of dentistry in 2021-2022. Methods: In a quasi-experimental study, 60 dentistry students of a class were assigned into two study (n=30) and control (n=30) groups by Permuted block randomization. In the case group, the skills were related to Fluoride therapy, fissure sealant therapy, and health education evaluated by DOPS. In the control group, these skills were evaluated by traditional evaluation methods. Each test was repeated three times. Finally, the satisfaction of students in the case group was assessed by a questionnaire. The chi-square test was used to compare qualitative variables. Repeated measure ANOVA test was used to compare the mean scores in three stages and two groups. P value less than 0.05 was considered significant. Data were analyzed using SPSS 16 software. Results: A significant difference in the mean score of Fluoride therapy, pit and fissure sealant therapy, and health education was seen between the case and control groups (P<0.001). Also a significant increase in these skills in the third stage of assessment in the case group was observed (P<0.001). The professors and students' satisfaction was considerably high on the DOPS test. Conclusion: The DOPS method had more impact on Fluoride therapy, pit and fissure sealant therapy, and health education's learning process in dentistry students than the conventional evaluation. The professors and students' satisfaction level was high regarding DOPS. The advantages of the DOPS method are student-centeredness, objectivity, and appropriate feedback.

Trainee anaesthetist self-assessment using an entrustment scale in workplace-based assessment.

The role of self-assessment in workplace-based assessment remains contested. However, anaesthesia trainees need to learn to judge the quality of their own work. Entrustment scales have facilitated a shared understanding of performance standards among supervisors by aligning assessment ratings with everyday clinical supervisory decisions. We hypothesised that if the entrustment scale similarly helped trainees in their self-assessment, there would be substantial agreement between supervisor and trainee ratings. We collected separate mini-clinical evaluation exercises forms from 113 anaesthesia trainee-supervisor pairs from three hospitals in Australia and New Zealand. We calculated the agreement between trainee and supervisor ratings using Pearson and intraclass correlation coefficients. We also tested for associations with demographic variables and examined narrative comments for factors influencing rating. We found ratings agreed in 32% of cases, with 66% of trainee ratings within one point of the supervisor rating on a nine-point scale. The correlation between trainee and supervisor ratings was 0.71, and the degree of agreement measured by the intraclass correlation coefficient was 0.67. With higher supervisor ratings, trainee ratings better correlated with supervisor ratings. We found no strong association with demographic variables. Possible explanations of divergent ratings included one party being unaware of a vital aspect of the performance and different interpretations of the prospective nature of the scale. The substantial concordance between trainee and supervisor ratings supports the contention that the entrustment scale helped produce a shared understanding of the desired performance standard. Discussion between trainees and supervisors on the reasoning underlying their respective judgements would provide further opportunities to enhance this shared understanding.

[Systematic analysis of evaluations of Objective Structured Clinical Exam (OSCE) multiple circuits: Explanatory variables and inter-rater correlations]. / Analyse systématique des évaluations de circuits multiples d'examen clinique objectif structuré (ECOS) : variables explicatives et corrélations inter-évaluateurs.

INTRODUCTION: Objective Structured Clinical Examinations (OSCEs) assess professional performance in a simulated environment. Following their integration into the reform of the 2nd cycle of medical studies (R2C), this pedagogical modality was implemented in France. This study investigates the variability of students' OSCE scores, as well as their inter-rater reproducibility. METHODS: This single-center retrospective study covered several sessions of evaluative OSCE circuits conducted between January 2022 and June 2023. Variables collected were: baseline situation family, competency domain, presence of a standardized participant for stations; gender and professional status for evaluators; scores (global, clinical and communication skills), number of previously completed OSCE circuits and faculty scores for students. RESULTS: The variability of the overall score was explained mainly (79.7%, CI95% [77.4; 82.0]) by the station factor. The student factor and the circuit factor explained 7.5% [12.9; 20.2] and<0.01% [2.10-13; 2.10-9] respectively. The inter-rater intra-class correlation coefficient was 87.2% [86.4; 87.9] for the global score. Station characteristics (starting situation, domain) and evaluator characteristics (gender, status) were significantly associated with score variations. CONCLUSION: This first study on the variability of OSCE circuit scores in France shows good reproducibility with influence of station characteristics. In order to standardize circuits, variability linked to the domain competency should be considered as well.

Impact of a university teaching of integrative medicine on the social representations of undergraduate medical students.

Integrative medicine, need to be inoffensive, effective, and of quality (World Health Organization). In 2010, the American Society of Teachers of Family Medicine approved 19 competencies for teaching integrative medicine to residents. In 2018, the University of Rennes created a course: "Integrative Medicine and Complementary Therapies". Up until then, the only feedback from the courses was the students' opinions. We investigated the impact on medical students' social representation.We performed a sociological analysis of students' social representations before and after the course. The social representation is based on the way an individual creates his or her universe of beliefs and ideas. After hearing, "What word or group of words comes to mind when you hear people speak of integrative medicine and complementary therapies?", students were asked to provide 5 words/phrases, rank their importance, and show their attitude towards these words/phrases. The frequency and importance of these words/phrases were used to construct social representations (with central cores, and primary and secondary peripheries) before and after the course.Among the 101 students registered, 59 provided complete responses before and 63 after the course. Before, the central core comprised "hypnosis" and "alternative medicine", while after: "complementary care" and "global care". We only identified first periphery before the course: "acupuncture" and "homeopathy". 4 new contrasting elements: "integration with conventional treatment", "patient's choice", "personalisation of care", and "caring relationship of trust".This teaching course positively affected students' social representation of integrative medicine, and might promote their use during future practices.

Relationship of emotional intelligence and capability of answering higher-order knowledge questions in physiology among first-year medical students.

Emotional intelligence (EI) has a positive correlation with the academic performance of medical students. However, why there is a positive correlation needs further exploration. We hypothesized that the capability of answering higher-order knowledge questions (HOQs) is higher in students with higher EI. Hence, we assessed the correlation between EI and the capability of medical students to answer HOQs in physiology. First-year undergraduate medical students (n = 124) from an Indian medical college were recruited as a convenient sample. EI was assessed by the Schutte Self-Report Emotional Intelligence Test (SSEIT), a 33-item self-administered validated questionnaire. A specially designed objective examination with 15 lower-order and 15 higher-order multiple-choice questions was conducted. The correlation between the examination score and the EI score was tested by Pearson's correlation coefficient. Data from 92 students (33 females and 59 males) with a mean age of 20.14 ± 1.87 yr were analyzed. Overall, students got a percentage of 53.37 ± 14.07 in the examination, with 24.46 ± 9.1 in HOQs and 28.91 ± 6.58 in lower-order knowledge questions (LOQs). They had a mean score of 109.58 ± 46.2 in SSEIT. The correlation coefficient of SSEIT score with total marks was r = 0.29 (P = 0.0037), with HOQs was r = 0.41 (P < 0.0001), and with LOQs was r = 0.14 (P = 0.19). Hence, there is a positive correlation between EI and the capability of medical students to answer HOQs in physiology. This study may be the foundation for further exploration of the capability of answering HOQs in other subjects.NEW & NOTEWORTHY This study assessed the correlation between emotional intelligence (EI) and the capability of medical students to answer higher-order knowledge questions (HOQs) in the specific context of physiology. The finding reveals one of the multifaceted dimensions of the relationship between EI and academic performance. This novel perspective opens the door to further investigations to explore the relationship in other subjects and other dimensions to understand why students with higher EI have higher academic performance.

The Development of the Canadian Physiotherapy Assessment of Clinical Performance (ACP) 2.0 - Alignment with the 2017 Competency Profile.

Purpose: Clinical education and assessment of students' performance during clinical placements are key components of Canadian entry-to-practice physiotherapy curriculum and important in developing entry-level physiotherapy practitioners. The Canadian Physiotherapy Assessment of Clinical Performance (ACP) is the measure currently used to assess physiotherapy student performance on clinical placements in most of the entry-to-practice physiotherapy programmes across Canada. The release of the 2017 Competency Profile by the National Physiotherapy Advisory Group resulted in a revision of the existing ACP. The purpose of this study is to report the process used to develop a revised version of the ACP based on the 2017 Competency Profile, henceforth called the ACP 2.0. Method: Using a multistage process, we sought input from Canadian clinical education academics, an expert consultant panel, as well as physiotherapists across Canada using a questionnaire, meetings, and an online survey, respectively. Results: Twelve of 15 clinical education academics responded to a questionnaire. The expert consultant panel (n = 12) met three times. There were 144 physiotherapists who initiated the national, online, survey and met the inclusion criteria; 84 completed the survey. In the ACP 2.0, rating scales and comments boxes were grouped, and additional text was added to 12 items for further clarification. The ACP 2.0 came to have 18 items and 9 comment boxes in addition to summative comments, in contrast to the original ACP's 21 items and 9 comment boxes. Conclusions: In November 2020, Canadian clinical education academics reviewed the proposed draft ACP 2.0 and unanimously accepted it for implementation in Canadian physiotherapy university programmes.

Objectif: l'enseignement clinique et l'évaluation du rendement des étudiants pendant les stages cliniques sont des éléments clés du programme canadien d'entrée en pratique de la physiothérapie et sont importants pour former des praticiens de la physiothérapie prêts à entrer en pratique. L'évaluation du rendement clinique de la physiothérapie au Canada (ÉPC) est la mesure actuellement en usage pour évaluer le rendement des étudiants en physiothérapie lors de leur stage clinique dans la plupart des programmes d'entrée en pratique de la physiothérapie au Canada. La publication du Profil des compétences par le Groupe consultatif national en physiothérapie en 2017 a donné lieu à une révision de l'ÉPC. La présente étude vise à rendre compte du processus utilisé pour mettre au point une version révisée de l'ÉPC d'après le Profil des compétences de 2017, désormais appelée l'ÉPC 2.0. Méthodologie: au moyen d'un processus échelonné, les chercheurs ont demandé l'apport d'universitaires canadiens en enseignement clinique, d'un groupe d'experts consultants et de physiothérapeutes des diverses régions du Canada dans le cadre d'un questionnaire, de réunions et d'un sondage en ligne, respectivement. Résultats: au total, 12 des 15 universitaires en enseignement clinique ont répondu à un questionnaire. Le groupe d'experts consultants (n = 12) s'est réuni trois fois. Enfin, 144 physiothérapeutes qui respectaient les critères d'inclusion ont entrepris le sondage national en ligne, et 84 l'ont terminé. Dans l'ÉPC 2.0, les échelles d'évaluation et les encadrés de commentaires ont été regroupés et du texte a été ajouté à 12 des points afin de les clarifier. L'ÉPC 2.0 comporte finalement 18 points et neuf encadrés de commentaires en plus des commentaires sommatifs, par rapport aux 21 points et aux neuf encadrés de commentaires de l'ÉPC original. Conclusions: en novembre 2020, les universitaires en enseignement clinique canadiens ont révisé le projet d'ÉPC 2.0 et en ont adopté la mise en œuvre à l'unanimité au sein des programmes universitaires de physiothérapie du Canada.

Translation and validation of the Persian version of the perception to care in acute situations (PCAS-P) scale in novice nurses.

BACKGROUND: Novice nurses providing care in acute conditions should have satisfactory performance. Accurate and appropriate evaluation of the performance of novice nurses in providing care in acute situations is essential for planning interventions to improve the quality of patient care. This study was conducted to translate and evaluate the psychometric properties of the Persian version of the Perception to Care in Acute Situations (PCAS-P) scale in novice nurses. METHODS: In this methodological study, 236 novice nurses were selected by the convenience sampling method. 17-item scale PCAS-P was translated into Persian by the forward-backward process. Then, this version was used for psychometric evaluation. For this purpose, face validity, content validity, and construct validity were assessed using confirmatory factor analysis. Internal consistency and stability reliability were calculated. The data were analyzed using SPSS and AMOS software. RESULTS: The PCAS-P scale maintained the meaning of the original English version and was clear, explicit, and understandable for novice nurses. Confirmatory factor analysis showed that this Persian version is consistent with the proposed model and confirmed the fit of the three-factor model. The values of Cronbach's alpha coefficient, McDonald's omega, Coefficient H, and average inter-item correlation were excellent for the overall scale and its dimensions, and the three latent factors had good convergent and discriminant validity. Additionally, the average measurement size was 0.944 ICC (95% CI 0.909 to 0.969). CONCLUSION: The PCAS-P scale is valid and reliable for measuring novice nurses' perception of acute situations.

A Multidimensional Model to Facilitate Within Person Comparison of Attributes.

In psychological research and practice, a person's scores on two different traits or abilities are often compared. Such within-person comparisons require that measurements have equal units (EU) and/or equal origins: an assumption rarely validated. We describe a multidimensional SEM/IRT model from the literature and, using principles of conjoint measurement, show that its expected response variables satisfy the axioms of additive conjoint measurement for measurement on a common scale. In an application to Quality of Life data, the EU analysis is used as a pre-processing step to derive a simple structure Quality of Life model with three dimensions expressed in equal units. The results are used to address questions that can only be addressed by scores expressed in equal units. When the EU model fits the data, scores in the corresponding simple structure model will have added validity in that they can address questions that cannot otherwise be addressed. Limitations and the need for further research are discussed.

Training future leaders: A qualitative study exploring leadership development in predoctoral dental curricula.

PURPOSE/OBJECTIVE: Commission on Dental Accreditation (CODA) accreditation standard 2-19 states predoctoral dental schools must assess students' ability to function successfully as the leader of the oral health care team. This study aimed to explore how dental schools incorporate leadership training into their curriculum to better understand the leadership skills students learn, the ways students engage in leadership training, and the opportunities students have to practice leadership skills with their peers. METHODS: The aim of this 2022 qualitative phenomenology study was to use semi-structured interviews with academic Deans at CODA-accredited dental schools and one subject matter expert to uncover types of cognitive, behavioral, and environmental factors influencing leadership training through the lens of social cognitive theory. All interviews were recorded on Zoom, transcribed, de-identified, and analyzed for recurring themes using NVivo. Eight academic Deans and one subject matter expert participated in the study. RESULTS: Four major themes emerged from the data: leadership is essential for dental professionals, leadership is incorporated into the curricula in diverse ways, students most often engage in leadership training opportunities with dental peers and interprofessional opportunities could be expanded, and dental schools often face barriers to incorporating leadership training. Vertically integrated case presentations and team-based practice management simulations are meaningful leadership development activities. Prominent barriers include time constraints, lack of faculty champions with teaching leadership expertise, and prioritizing the development of hand skills. CONCLUSION: Standard practices for student leadership development and assessment do not appear to exist across dental school curricula. Findings support the need for a leadership development framework.

Performance of ChatGPT on Stage 1 of the Taiwanese medical licensing exam.

Introduction: Since its release by OpenAI in November 2022, numerous studies have subjected ChatGPT to various tests to evaluate its performance in medical exams. The objective of this study is to evaluate ChatGPT's accuracy and logical reasoning across all 10 subjects featured in Stage 1 of Senior Professional and Technical Examinations for Medical Doctors (SPTEMD) in Taiwan, with questions that encompass both Chinese and English. Methods: In this study, we tested ChatGPT-4 to complete SPTEMD Stage 1. The model was presented with multiple-choice questions extracted from three separate tests conducted in February 2022, July 2022, and February 2023. These questions encompass 10 subjects, namely biochemistry and molecular biology, anatomy, embryology and developmental biology, histology, physiology, microbiology and immunology, parasitology, pharmacology, pathology, and public health. Subsequently, we analyzed the model's accuracy for each subject. Result: In all three tests, ChatGPT achieved scores surpassing the 60% passing threshold, resulting in an overall average score of 87.8%. Notably, its best performance was in biochemistry, where it garnered an average score of 93.8%. Conversely, the performance of the generative pre-trained transformer (GPT)-4 assistant on anatomy, parasitology, and embryology was not as good. In addition, its scores were highly variable in embryology and parasitology. Conclusion: ChatGPT has the potential to facilitate not only exam preparation but also improve the accessibility of medical education and support continuous education for medical professionals. In conclusion, this study has demonstrated ChatGPT's potential competence across various subjects within the SPTEMD Stage 1 and suggests that it could be a helpful tool for learning and exam preparation for medical students and professionals.

[Living Networks in Health during the covid-19 pandemic: Evaluation of the multiprofessional training course "Nós da Linha de Frente" in Manaus, Amazonas, Brazil]. / Redes vivas en salud en la pandemia de covid-19: evaluación del curso de formación multiprofesional "Nós da Linha de Frente" en Manaus, Amazonas, Brasil.

This article presents the evaluation of the therapeutic intervention course in situations of traumatic loss and grief, offered online by the Universidade do Estado do Amazonas, between April and August 2021, for volunteer professionals. A semi-structured questionnaire with open and closed questions was administered. The analysis of the 55 responses from volunteer professionals and 14 responses from individuals who received the intervention was based on the Kirkpatrick evaluation model. The results demonstrate the theoretical and practical success of the course, with a positive assessment at the reaction, learning, behavior, and results levels, emphasizing collective work and the motivation of the course speakers. The main acquired knowledge pertained to topics related to death and grief care. The behaviors resulting from the course fall within the affective-social and technical-professional domains. Regarding impact, there was a predominance of satisfaction among the individuals who received the intervention, as well as identification of changes brought about by the training.

Este artículo presenta la evaluación del curso de intervención terapéutica en situaciones de pérdida traumática y duelo, ofrecido en línea por la Universidade do Estado do Amazonas, entre abril y agosto de 2021, para profesionales voluntarios. Se aplicó un cuestionario semiestructurado con preguntas abiertas y cerradas. El análisis de las 55 respuestas de profesionales voluntarios y 14 respuestas de personas atendidas se basó en el modelo de evaluación Kirkpatrick. Los resultados muestran el acierto teórico y práctico del curso, con una evaluación positiva en los niveles de reacción, aprendizaje, comportamiento y resultados, en la que se destacó el trabajo colectivo y la motivación de los disertantes del curso. Los principales contenidos aprehendidos fueron temas relacionados con el cuidado de la muerte y el duelo. Las conductas derivadas del curso se encuadran en los campos afectivo-social y técnico-profesional. En cuanto al impacto, hubo un predominio de la satisfacción de las personas atendidas, así como identificación de cambios provocados por la formación.

Can peer feedback substitute for faculty feedback in predoctoral dental education?

PURPOSE/OBJECTIVES: The ability to give and receive feedback is a key skill to develop during predoctoral dental education, and the use of peer feedback specifically offers distinct benefits including a different understanding of material due to peers' proximity of knowledge development and assisting with overburdened instructors. However, it is unclear if peer feedback offers similar quality to instructor feedback. METHODS: Dental students in two different graduation years provided quantitative and qualitative peer feedback on a case-based oral and maxillofacial pathology simulation. The data from these exercises were aggregated and analyzed to compare the quality of qualitative feedback to course examination scores. Student perceptions of peer feedback were also recorded. RESULTS: The mean quality of feedback was not correlated with course examination scores, though the number of times students gave high-quality feedback and received high-quality feedback was correlated with course examination scores. Student feedback overall had a lower quality than instructor feedback, though there was no significant difference between instructor feedback quality and the maximum student feedback quality received. Student perceptions of the utility of feedback were positive. CONCLUSION: While instructor feedback is more reliable and consistent, our findings suggest that in most instances, at least one peer in moderate-sized groups is able to approximate the quality of instructor feedback on case-based assignments.

Using the Delphi method to develop a clinical skills assessment tool for advanced practice nurses in psychiatry and mental health / Élaboration d'un outil d'évaluation des compétences en stage des étudiants infirmiers en pratique avancée, mention psychiatrie et santé mentale à partir d'une méthode Delphi

Introduction: In France, advanced practice for psychiatric and mental health nurses has been developing since 2019. The acquisition of clinical skills by advanced practice nursing students requires monitoring and evaluation. This article outlines the process for creating an internship logbook. Objective: To develop and verify a tool for assessing the skills of advanced practice nursing students specializing in psychiatry and mental health during the internship stage. Method: The Delphi method described by Hasson was used, with the aim of obtaining a consensus of over 80% of the experts consulted. Results: A panel of ten experts, all internship tutors at partner healthcare establishments of Aix Marseille University, was consulted twice. The internship logbook was composed of 68 items. Discussion: This tool is the product of a multidisciplinary process including nurses and psychiatric nurses. It can be adapted to a variety of learning environments and helps tutors track the acquisition of clinical skills during the internships, both through external and self-assessment. Conclusion: A new Delphi consultation will foster uptake of this tool. Advanced practice nurses in psychiatry and mental health will be included in the group of experts.

Una revisión sistemática del uso de la inteligencia artificial en la educación / A systematic review of the use of artificial intelligence in education

Introducción. El uso de la inteligencia artificial (IA) en la educación ha sido objeto de una creciente atención en los últimos años. La IA se ha utilizado para mejorar la personalización del aprendizaje, la retroalimentación y la evaluación de los estudiantes. Sin embargo, también hay desafíos y limitaciones asociados. El objetivo de este trabajo fue identificar las principales tendencias y áreas de aplicación de la inteligencia artificial en la educación, así como analizar los beneficios y limitaciones de su uso en este ámbito. Métodos. Se llevó a cabo una revisión sistemática que exploró el empleo de la inteligencia artificial en el ámbito educativo. Esta revisión siguió una metodología de investigación basada en la búsqueda de literatura, compuesta por cinco etapas. La investigación se realizó utilizando Scopus como fuente de consulta primaria y se empleó la herramienta VOSviewer para analizar los resultados obtenidos. Resultados. Se encontraron numerosos estudios que investigan el uso de la IA en la educación. Los resultados sugieren que la IA puede mejorar significativamente la personalización del aprendizaje, proporcionando recomendaciones de actividades y retroalimentación adaptadas a las necesidades individuales de cada estudiante. Conclusiones. A pesar de las ventajas del uso de la IA en la educación, también hay desafíos y limitaciones que deben abordarse, como la calidad de los datos utilizados por la IA, la necesidad de capacitación para educadores y estudiantes, y las preocupaciones sobre la privacidad y la seguridad de los datos de los estudiantes. Es importante seguir evaluando los efectos del uso de la IA en la educación para garantizar su uso efectivo y responsable.

Introduction. The use of artificial intelligence (AI) in education has been the subject of increasing attention in recent years. AI has been used to improve personalized learning, feedback, and student assessment. However, there are also challenges and limitations. The aim of this study was to identify the main trends and areas of application of artificial intelligence in education, as well as to analyze the benefits and limitations of its use in this field. Methods. A systematic review was carried out on the use of artificial intelligence in education, using a literature search research methodology with five stages, based on the Scopus query and the tool for analyzing results with VOSviewer. Results. Numerous studies investigating the use of AI in education were found. The results suggest that AI can significantly improve personalized learning by providing activity recommendations and feedback tailored to the individual needs of each student. Conclusions. Despite the advantages of using AI in education, there are also challenges and limitations that need to be addressed, such as the quality of data used by AI, the need for training for educators and students, and concerns about the privacy and security of student data. It is important to continue evaluating the effects of AI use in education to ensure its effective and responsible use.

Preliminary Development of a Program ABSITE Dashboard (PAD) to Guide Curriculum Innovation.

PURPOSE: Medical Knowledge for general surgery residents' is assessed by the American Board of Surgery In- Training Examination (ABSITE). ASBITE score reports contain many metrics residency directors can utilize to assess resident progress and perform program evaluation. The purpose of this study was to develop a framework to evaluate program effectiveness in teaching specific subtest and subtopic areas of the ABSITE, using ABSITE score reports as an indicator. The aim is to demonstrate the identification of topic areas of weakness in program-wide performance on the ABSITE to guide proposed modification of the general surgery residency program curriculum, and to initiate development of a data visualizing dashboard to communicate these metrics. METHODS: A single institution retrospective study was performed utilizing ABSITE score reports from general surgery residents at a large academic training program from 2017 to 2020. ABSITE performance metrics from 320 unique records were entered into a database; statistical analysis for linear trends and variance were conducted for standard scores, subtest standard scores, and incorrect subtest topics. Deviation from national average scores were calculated by subtracting the national average score from each subtest score for each trainee. Data were displayed as medians or proportions and are displayed to optimize visualization as a proof-of-concept for the development of a program dashboard. RESULTS: Trends and variance in general surgery program and cohort performance on various elements of the ABSITE were visualized using figures and tables that represent a prototype for a program dashboard. Figure A1 demonstrates one example, in which a heatmap displays the median deviation from national average scores for each subtest by program year. Boxplots show the distribution of the deviation from national average, range for national average scores, and the recorded scores for each subtest by program year. Trends in median deviation from the national average scores are displayed for each program year paneled by subtest or for each exam year paneled by cohort. Median change in overall test scores from one program year to another in a cohort is visualized as a table. Bar graphs show the most often missed topics across all program years and heatmaps were generated showing the proportion of times each topic was missed for each subtest and exam year. CONCLUSIONS: We demonstrate use of ABSITE reports to identify specific thematic areas of opportunities for curriculum modification and innovation as an element of program evaluation. In this study we demonstrate, through data analysis and visualization, feasibility for the creation of a Program ABSITE Dashboard (PAD) that enhances the use of ABSITE reports for formative program evaluation and can guide modifications to surgery program curriculum and educational practices.

Interrater reliability in neurology objective structured clinical examination across specialties.

PURPOSE: To assess interrater reliability and examiners' characteristics, especially specialty, associated with scoring of neurology objective structured clinical examination (OSCE). MATERIAL AND METHODS: During a neurology mock OSCE, five randomly chosen students volunteers were filmed while performing 1 of the 5 stations. Video recordings were scored by physicians from the Lyon and Clermont-Ferrand university teaching hospitals to assess students performance using both a checklist scoring and a global rating scale. Interrater reliability between examiners were assessed using intraclass coefficient correlation. Multivariable linear regression models including video recording as random effect dependent variable were performed to detect factors associated with scoring. RESULTS: Thirty examiners including 15 (50%) neurologists participated. The intraclass correlation coefficient of checklist scores and global ratings between examiners were 0.71 (CI95% [0.45-0.95]) and 0.54 (CI95% [0.28-0.91]), respectively. In multivariable analyses, no factor was associated with checklist scores, while male gender of examiner was associated with lower global rating (ß coefficient = -0.37; CI 95% [-0.62-0.11]). CONCLUSIONS: Our study demonstrated through a video-based scoring method that agreement among examiners was good using checklist scoring while moderate using global rating scale in neurology OSCE. Examiner's specialty did not affect scoring whereas gender was associated with global rating scale.

Gender differences in self-assessed performance and stress level during training of basic interventional radiology maneuvers.

OBJECTIVES: Gender differences have been reported to influence medical training. We investigated gender differences encountered during training in interventional radiology maneuvers. METHODS: Catheter handling was analyzed under standardized conditions in 64 participants naïve to endovascular procedures (26 women, 38 men). Objective (e.g., catheter pathway, catheter movements, required time) and subjective parameters (stress level) were recorded. The NASA-Task Load Index (NASA-TLX; 1-20 points) was used to assess participants' stress levels and perceived workload. RESULTS: In the easier tasks, no significant differences between male and female participants regarding catheter handling were observed. In the most complex task, female participants took themselves more time (688 ± 363 vs. 501 ± 230 s; p = 0.02), asked for help more frequently (n = 19 vs. n = 8) and earlier than men (203 ± 94 vs. 305 ± 142 s; p = 0.049), whereas men stood out by more agitated catheter handling (6.0 ± 1.8 vs. 4.8 ± 1.6 movements/s; p = 0.005). Overall, female participants perceived tasks to be more difficult (11.5 ± 4.2 vs. 9.6 ± 3.3; p = 0.016), perceived higher stress levels (8.9 ± 4.9 vs. 6.3 ± 4.4; p = 0.037), and rated their own performance lower (9.12 ± 3.3 vs. 11.3 ± 3.3; p = 0.009). However, female participants were able to correlate self-assessed with objective parameters correctly (r between -0.555 and -0.469; p = 0.004-0.018), whereas male participants failed to correctly rate their performance (p between 0.34 and 0.73). Stress levels correlated with objective parameters in males (r between 0.4 and 0.587; p < 0.005), but not in female participants. CONCLUSION: Perceived stress levels, self-evaluation skills, and catheter handling differ greatly between untrained male and female participants trying to solve interventional radiological tasks. These gender-specific differences should be considered in interventional radiology training. CLINICAL RELEVANCE STATEMENT: As psychological aspects may influence individual working strategies gender-specific differences in self-perception while learning interventional radiology maneuvers could be essential regarding success in teaching and treatment outcomes. KEY POINTS: • After performing standardized training, 38 male and 26 female volunteers showed significant differences regarding objective and self-assessed performance, as well as in perceived workload while performing simulated endovascular catheter maneuvers. • After solving simulated endovascular radiological tasks, female participants were able to self-assess their objective performance much more accurately than male participants. • Women took more time to solve simulated endovascular tasks and asked earlier and more frequently for help than men.

Twelve tips to aid interpretation of post-assessment psychometric reports.

Post-assessments psychometric reports are a vital component of the assessment cycle to ensure that assessments are reliable, valid and fair to make appropriate pass-fail decisions. Students' scores can be summarised by examination of frequency distributions, central tendency measures and dispersion measures. Item discrimination indicies to assess the quality of items, and distractors that differentiate between students achieving or not achieving the learning outcomes are key. Estimating individual item reliability and item validity indices can maximise test-score reliability and validity. Test accuracy can be evaluated by assessing test reliability, consistency and validity and standard error of measurement can be used to measure the variation. Standard setting, even by experts, may be unreliable and reality checks such as the Hofstee method, P values and correlation analysis can improve validity. The Rasch model of student ability and item difficulty assists in modifying assessment questions, pinpointing areas for additional instruction. We propose 12 tips to support test developers in interpreting structured psychometric reports, including analysis and refinement of flawed items and ensuring fair assessments with accurate and defensible marks.